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ABSTRACT 



A method for increasing compression efficiency of a stereo- 
scopic image encoder employed for compressive ly encoding 
first and second signals representing respective first and 
second views of a stereoscopic image, comprises: providing 
a first histogram of the first view and a second histogram of 
the second view, the first and second histograms represent- 
ing a frequency of occurrence of a characteristic component 
of the first and second views, respectively; comparing the 
first histogram of the first view with the second histogram of 
the second view and estimating the mismatch in gain, a, and 
mismatch in offset, b, between the first and second views; 
correcting the stereoscopic image by multiplying the gain a 
and the second signal to obtain a gain corrected second 
signal for the stereoscopic image and adding the offset b to 
the gain corrected second signal to obtain a gain and offset 
corrected second signal, the gain and offset corrected second 
signal being obtained prior to encoding to increase encoding 
efficiency. The quality of the resulting stereoscopic image 
formed from decoding the first signal and gain and offset 
corrected second signal is greatly improved. 

50 Claims, 9 Drawing Sheets 
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GAIN AND OFFSET CORRECTION FOR 
EFFICIENT STEREOSCOPIC CODING AND 
IMPROVED DISPLAY 

RELATED APPLICATIONS 

This application claims the benefit of provisional U.S. 
Patent Application Ser. No. 60/009,315 filed Dec. 29, 1995. 

FIELD OF THE INVENTION 

This invention relates generally to stereoscopic and multi- 
view video and display systems, and more particularly, to a 
method and apparatus for digitally correcting mismatch in 
gain and offset parameters to correct color and brightness 
differences between two views of stereoscopic video. The 
method and apparatus improves not only the coding effi- 
ciency of resulting corrected views forming stereoscopic 
video but, also the displayed quality of stereoscopic video. 

The techniques of this invention can be particularly 
advantageous in improving the coding efficiency in compat- 
ible stereoscopic coding, where one view of stereoscopic 
video called the compatible view is coded independently, 
whereas the other view is coded dependently with respect to 
the compatibly coded view. 

BACKGROUND OF THE INVENTION 

Although the human visual system can naturally capture 
the three dimensional world, most imaging and display 
systems limit our abilities by presenting only a two dimen- 
sional (2D) mapping of the three dimensional (3D) world. 
Many current as well as emerging applications can signifi- 
cantly benefit from the high degree of realism provided by 
3D scenes with depth. In fact, human ability to perceive 
realism in a scene is directly related to our ability to perceive 
depth accurately in a scene. A real 3D scene can be recon- 
structed from many 2D views of a scene imaged from 
different perspectives. Such a representation would not only 
allow the perception of depth in a scene but look-around 
capability as well. However, due to practical display 
constraints, although only a less complex solution employ- 
ing two views of a scene is easily realizable, it can still 
impart the sensation of depth. These two views are required 
to be imaged under specific constraints and specifically, one 
view is intended for each respective eye of a human visual 
system so that a human brain can generate the depth infor- 
mation necessary to perceive realism. The two views 
together represent stereoscopic video where each view is 
similar to normal video except for the fact that the two views 
are related under the constraints imposed by stereoscopic 
vision. Under specified constraints, the two views imaging a 
scene differ by what is known as disparity between the 
views, which is typically only a few pixels in vertical 
direction, but can be of the order of about 40 pixels or higher 
in the horizontal direction, assuming each view is imaged at 
normal TV resolution. 

Stereoscopic video has potential applications in 
education, training, 3D movies/entertainment, medical 
surgery, videoconferencing, virtual travel and shopping, 
multimedia presentations, video games and immersive vir- 
tual reality experiences, and others. Although, the potential 
applications of 3D/stereoscopic video are many, there arc 
several challenges to be overcome before its potential can be 
truly harnessed and its use becomes wide spread. Currently 
the two primary challenges are: a convenient stereoscopic/ 
3D display, and, a highly efficient compatible coding 
scheme. Regarding the issue of displays, most practical 
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means of displaying stereoscopic/3 D video requires viewers 
to wear specialized viewing glasses. These viewing glasses 
may be active shuttered glasses, which contain electronics, 
or passive polarizing glasses, which are somewhat less 

5 cumbersome. Although some displays not requiring special- 
ized viewing glasses (autostereoscopic systems) are 
available, they impose other restrictions, e.g., viewing zones 
and discreteness of views and may typically require between 
10 and 20 views for realism. Stereoscopic video, on the other 

1Q hand, although it requires use of specialized glasses, can 
impart perception of depth in a scene and uses only two 
views, left-view intended for the left-eye and right-view 
intended for right-eye of a human visual system in either 
time-sequential (with active synchronized shuttered glasses) 

15 or time-simultaneous (with passive polarizing glasses). 
Besides the display issue, the other main issue is that of 
efficient digital compression of 3D/stereoscopic video so 
that the multiple views can be easily manipulated, stored or 
transmitted as needed. Towards that end, interworking with 

2Q existing or emerging standards based coding schemes as 
well as existing displays for normal video is a necessity in 
many applications. Of particular relevance is the second 
phase of ISO Moving Pictures Experts Group (MPEG-2) 
video coding standard, which offers a good solution to a 

^ large variety of applications requiring digital video includ- 
ing broadcast TV via satellite, cable TV, HDTV, digital 
VCRs, multipoint video and others. 

As is well known, techniques based on or extending from 
basic MPEG-2 video coding increases coding efficiency. 

30 Basic video coding in MPEG-2 involves motion- 
compensated DCT coding of frame- or field-pictures and is 
dealt with in detail in A. Puri, "Video Coding Using the 
MPEG-2 Compression Standard," Proceedings of SPIE 
Visual Communications and Image Processing, Boston, 

35 Mass., November 1993, pp. 1701-1713, and in R. L. 
Schmidt, A. Puri and B. G. Haskell, "Performance Evalua- 
tion of Nonscalable MPEG-2 Video Coding," Proceedings 
of SPIE Visual Communications and Image Processing, 
Chicago, ILL, September 1994, pp. 296-310 the contents and 

40 disclosure of both references which are expressly incorpo- 
rated by reference herein. 

In the past, several attempts have been made to reduce the 
bandwidth of digital stereoscopic video. Among the more 
promising methods presented recently are the methods 

45 which are based on or are extensions of MPEG-2 video 
coding. These methods typically employ either compensa- 
tion of disparity on block by block basis between the two 
views or use both motion and disparity compensation, also 
on a block by block basis. However, it has been determined 

50 that for some stereoscopic video scenes, disparity compen- 
sation does not work very well, attributable to significant 
global differences in brightness and in color between the two 
views of a stereoscopic scene. 

It would therefore be highly desirable to correct for 

55 differences in brightness and in color between the two views 
of a stereoscopic scene and perform the corrections globally 
for the sake of compatibility with MPEG-2 based coding of 
stereoscopic video. 
It would also be highly desirable to correct for global 

60 mismatch in gain and offset between the left- and the 
right-views of stereoscopic video due to differences in 
imaging cameras, the differences in imaging sensors of 
cameras, differences in brightness and color balance adjust- 
ments of the sensors etc. Moreover, separate mismatch 

65 correction for global gain and offset for each of the three 
component signals, luminance, Y, and color signals, Cr and 
Cb, would be highly desirable. 
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SUMMARY OF THE INVENTION 

This invention is primarily concerned with the issue of 
digital correction of mismatch between views of stereo- 
scopic video resulting in highly efficient compression and 5 
improved quality of displayed views. Besides application of 
stereoscopic views, the techniques of this invention can be 
easily extended to coding of more than two views (here, 
referred to as multi-views). While it is possible to encode 
each of the two views of stereoscopic video separately 
(simulcast), a combined (joint) coding of two views is 
usually more efficient, since the two views are very related 
and contain significant redundancies which can be exploited 
for compression. It is possible to achieve joint coding of 
stereoscopic video by two basic approaches: The first 
approach results in compatibility with normal video in the 
sense that one view of stereoscopic video may be decoded 
for normal video display, while both views could be decoded 
for stereoscopic display. The second approach involves joint 
coding without regard to compatibility with monoscopic 2Q 
video, and presumably a higher degree of compression may 
be achieved. Any of the two coding approaches can be used 
with the instant invention. 

Furthermore, the invention increases the coding efficiency 
of earlier proposed techniques based on or extending from 25 
basic MPEG-2 video coding, and thus, offers a compatible 
solution for future digital broadcast 3DTV. For instance, 
temporal scalability in MPEG-2 involves coding of video as 
two layers in time, such that the first layer, called base-layer, 
can be decoded independent of the second layer, called the 30 
enhancement -layer. The base-layer can be coded with any 
coder, such as, motion compensated DCT coders of H.261, 
MPEG-1 or basic MPEG-2, while the enhancement layer 
also uses the motion compensated DCT structure but with 
temporal prediction from the base -layer. 35 

Thus, the inventive high level compatible stereoscopic 
codec structure is flexible with respect to the coders that may 
be used for independently and dependently coded views. For 
the sake of an example, assume the left- view is coded 
independently and the right-view is coded dependently. 40 
Now, the left-view could be coded by a nonscalable 
MPEG-2 coder, a main-profile coder, whereas the right-view 
could be coded with an MPEG-2 Temporal Enhancement 
Encoder, like the one used in Temporal Scalability. In fact, 
in the ongoing work in MPEG-2 on stereoscopic video 45 
coding, a stereoscopic video coding structure derived from 
temporal scalability as described in Puri, L. Yan and B. G. 
Haskell, "Temporal Resolution Scalable Video Coding," 
Proceedings of IEEE International Conference on Image 
Processing, Austin, Tex., November 1994, pp. 947-951 and 50 
A. Puri, R. V. Kollarits and B. G. Haskell, "Stereoscopic 
Video Compression using Temporal Scalability" Proceed- 
ings of SPIE Visual Communications and Image Processing, 
Taipei, Taiwan, May 1995, the contents and disclosure of 
both references which are expressly incorporated by refer- 55 
ence herein, may be used. This subject matter is additionally 
described in co-pending U.S. patent application Ser. No. 
08/452,463, entitled "Digital 3-D Stereoscopic Video Com- 
pression Technique Utilizing Disparity and Motion Com- 
pensated Predictions" and co-pending U.S. patent applica- 60 
tion Ser. No. 08/452,464, entitled "Digital 3-D Stereoscopic 
Video Compression Technique Utilizing Disparity Esti- 
mates" the contents and disclosure of both applications are 
expressly incorporated by reference herein. Thus, although 
examples are provided herein using MPEG-2 video coding 65 
context, the invention is equally relevant to currently ongo- 
ing work in MPEG-4. Another example, more applicable in 
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the future, may use basic MPEG-2 video coding for left- 
view and perhaps an MPEG-4 coder optimized for disparity/ 
motion compensated coding of right-view. Alternatively, an 
ITU-T's H.263 based coder or a basic MPEG-4 coder may 
be used for left- view with right -view coded using an opti- 
mized MPEG-4 utilizing disparity/motion compensated cod- 
ing. 

The invention particularly relates to improvements in 
methods for digital compression of stereoscopic/3D video 
by correcting for mismatch in imaging parameters such as, 
gain and offset differences between the two views which 
constitute stereoscopic video. In particular, gain and offset 
differences not only for luminance but also for color com- 
ponents are corrected; thus, mismatch in brightness and/or 
color balance between the two views of a scene due to 
differences in imaging parameters is rectified. This not only 
leads to increase in compression efficiency when coding 
stereoscopic video but also results in improvements in 
quality of displayed stereoscopic video. 

For stereoscopic scenes, one view of the scene can be 
used as the reference signal and digital correction of the 
other view for mismatch needs to be performed. This 
correction for mismatch allows improved prediction 
between the views and thus contributes to overall coding 
efficiency. Typically, compatible coding is performed allow- 
ing either standalone decoding of one layer for display on 
normal displays or decoding of both layers, the standalone 
layer and the layer coded with respect to it, for display on 
stereoscopic displays. 

Three example methods for estimation of gain and offset 
mismatch parameters between left and right views are 
described. They differ in assumptions that they make regard- 
ing uniqueness of characteristics of histograms, stationarity 
of image statistics and geometrical matching of shapes of 
histograms. The first method involves determination of 
unique characteristics in both both left -view and right-view 
histograms and works well when such unique characteristics 
can be identified. The second and third methods are more 
general with the second method relying purely on linear 
minimization of mean square error estimates between gain 
corrected right-view signal and left-signal, and the third 
method using geometrical appearance of histograms for 
matching and relying on first and second order moments of 
histograms. It turns out that although the underlying 
assumptions in gain and offset estimation in the three 
methods are somewhat different, they rely on the same basic 
approach for gain and offset correction. 

In the basic scheme described herein, from among the two 
views constituting stereoscopic video, one view, e.g., the 
left-view, is used as reference. The other view, e.g., the 
right -view, is then corrected for gain and offset mismatch 
independently on the luminance and the two chrominance 
components. In the first approach, mismatch correction is 
applied globally prior to coding and thus compatibility with 
MPEG-2 Temporal Scalability based stereoscopic coding is 
maintained. This approach not only results in increased 
coding efficiency when using a two layer Temporal scalabil- 
ity based codec structure, but also results in improvement of 
displayed quality due to reduction in mismatches between 
left- and right-views. 

In another approach, mismatch estimation and correction 
may be performed at the decoder as a postprocessing opera- 
tion prior to display and not prior to coding; this improves 
only the display quality but does not improve the coding 
efficiency. Sometimes, to minimize complexity of decoder/ 
display processing, it may not be practical to estimate 
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mismatch at the decoder/display processor, while correction FIG. 11A shows a high level block diagram of a Stereo- 

of mismatch which does not require significant computa- scopic Video Codec employing Global Gain and Offset 

tional complexity may be possible at the decoder. In these Estimation and Correction Postprocessing, arranged in 

instances, gain and offset estimation may be performed at accordance with the invention. 

the encoder and these global parameters which require as FIG. 11B shows a high level block diagram of a Stereo- 
minimal overhead may be transmitted either at scene cuts, scopic Video Codec employing Global Gain and Offset 
group-of-pictures, or on a picture basis in the bitstream. Estimation Preprocessing and Global Gain and Offset Cor- 
These parameters would be extracted by the decoder which rection Postprocessing, arranged in accordance with the 
provides them to postprocessor for correction. invention. 

Although examples are presented primarily in context of 10 
gain and offset correction for MPEG-2 based compatible 
stereoscopic coding, the generalized codec structure 

described is equally applicable to currently ongoing work in We now describe in detail, the framework, apparatus and 

MPEG-4 on stereoscopic/multi-view coding. methods of this invention to yield highly efficient compres- 

15 sion of stereoscopic video while preserving compatibility 

BRIEF DESCRIPTION OF THE DRAWINGS with normal video displays as well as improving the quality 

Reference may now be had to a preferred embodiment of of dis P la y cd stereoscopic video, 

the gain and offset correction apparatus and method of the Method 1 

invention, taken in conjunction to the accompanying _ . , . ,. . ... ■ , \ 

drawin in which- This technique applies to images having histograms with \ 

„ „ , . . , at least two uniquely identifiable points with significantly \ 

1 shows a simplified diagram of principle of stereo- diffcrcnt a lit udes, for ex le> « dark » and « very \ 

scopic imaging, gain corrected compression, and display, m brighl » contcnts Such pointsarc usually found in re d 7 

accordance with the invention. bhie md iumiQanoc (y) histograms and less frequently in J 

FIG. 2 shows a high level block diagram of a Stereoscopic 25 histogram of color components such as, Cb and Cr. 

Video Codec employing Global Gain and Offset Estimation Examples of well behaved histograms for left- and right- 

and Correction Preprocessing, arranged in accordance with views of stereoscopic video are respectively shown in FIG. 

the invention. 4A and FIG. 4B. The important features of these histograms 

FIG. 3 shows an example block diagram of an MPEG-2 are that the maximum and minimum levels are not populated 

based Stereoscopic Video Codec employing Global Gain 30 or arc very sparsely populated and well defined end points^N 

and Offset gain and offset first method for Corrected Pre- and peak points exist. Using the left-view as reference, gain 1 

processing arranged in accordance with the invention. and offset values that must be applied to the right-view to | 

FIG. 4A shows an example of well behaved histogram of correct for mismatch can be obtained by solving two simul- V 

left-view of stereoscopic video. taneous equations utilizing a pair of unique matching points I 

FIG. 4B shows an example of well behaved histogram of 35 m ^ two histograms, 

right-view of stereoscopic video requiring correction. Let x li ™ d X L2 be ^° unK i ue P oints on left-view 

FIG. 5 shows a detailed block diagram of Global Gain and ™f ^ T&m *" d f the ~«SPO«^8 ™ lchi »f P<*>* °" 

OffsetEsUmationPreproc^ssorbasedonthecomputationof ^t-view image histogram be x R1 and x^. If gain and 

• nr j j - j -*u *u • *• offset values of the right -view image with respect to the 

gam and offset, arranged in accordance with the invention. 40 , A . . & , , Jf . \ . ^ 

& & 4U left-view image are given by a and b, respectively, the two 

FIG. 6 shows a detailed flowchart for determining a pair simultaneous equations of interest are X^-ax^+b and 

of unique points needed for calculation of global gain and x ^ajfe+b. Rewriting the second equation, offset is given 

offset, arranged in accordance with the invention. as b-x^-ax^. Substituting this equation in the first equa- 

FIG. 7 shows a detailed block diagram of Global Gain and tion results in gain 

Offset Correction Preprocessor using gain and offset 45 

parameters, arranged in accordance with the invention. x u -xu 

FIG. 8 shows a detailed block diagram of Global Gain and " - x B2 ' 
Offset Estimation Preprocessor based on the second method 
for computation of gain and offset, arranged in accordance 

with the invention. 50 Method 2 

FIG. 9A shows a detailed block diagram for calculating Therc arc otQer gcneral techniquc5 that are more appli- 

the First Moment, l 1L , needed in estimation of global gain cable and un]ike Method 1 do not require identification of 

and offset, arranged in accordance with the invention. uniquc feature points tnat match in left- and right-view 

FIG. 9B shows a detailed block diagram for calculating 5S histograms. One such method is based on use of linear 

the First Moment, 1^, needed in estimation of global gain regression to minimize mean square error. Let a be the gain 

and offset, arranged in accordance with the invention. an j b be the offset of right-view image with respect to 

FIG. 9C shows a detailed block diagram for calculating left-view image used as reference. Then assuming random 

the Second Moment, I^, needed in estimation of global gain variables, a generalized approach for estimating a random 

and offset, arranged in accordance with the invention. 60 variable by a function of another random variable can be 

FIG. 9D shows a detailed block diagram for calculating used and the problem of determining a and b can be reduced 

the Joint Moment, 1^, needed in estimation of glohal gain to mean square estimation. In particular, an estimation of 

and offset, arranged in accordance with the invention. left-view can be made from the right-view as x L =ax R +b, 

FIG. 10 shows a detailed block diagram of Global Gain where the estimation error can be written as e= 

and Offset Estimation Preprocessor based on the third 65 x J ^=(x L -(ax R +b)). 

method for computation of gain and offset, arranged in The mean square error is computed as the expected value 

accordance with the invention. of the square of error, that is E(e 2 )=^-E(x i 2 +(ax 7i +b) 2 -2x z , 
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(ax^+b)). Solving this equation results in £=E(x A 2 )+a a E Equating the second moments, E(x^ 2 )=E(ax >? +b) 2 , and 

(x^ )+b 4-2abE(x^)-2aE(x i x /t )-2bE(x i ). To minimize mean solving this equation by recognizing that expectation E( ) is 

square error, a partial derivative with respect to a is first a linear operation, results in E(x i 2 )=a 2 E(x R )+2abE(x i! )+b 2 . 

computed and set to zero: Substituting b from the offset equation above into this 

5 equation, results in E(x z , 2 )«a 2 E(x A 2 )+2aE(x J? )[E(x £ )-aE 

d c (X/^MECXiVaEfx^)] 2 . Further solving this equation results 

0 = £ = oE(4) + bE{x R ) - E(x LXR ), or in E(x f 2 )=a f E(x i 2 )+(E(x i )) 2 -a 2 (E(x^) 2 . Finally, gain a can 

_ be derived by rewriting this equation: 

bE(x R ) = E{x L x R )-aE{x 2 lt ). 



f 

Next, a partial derivative with respect to b is computed a = I — -r ■ 

and set to zero: \ * R 

0 = ^ = b + aEM - Efr L ), or b = E(x L ) - a£(x R ). _ A S ain > replacing expectation E( ) by appropriate statistical 

9b 15 averages as in method 2, we get, 

By combining the two partial derivative equations a new / fit - CilJ 2 

expression for a results: a- a = yj / M _(/ 1Jt jJ * 

E(x L x R )-E(x L )E(x R ) 20 

a = E(x 2 )~{E{x R )) 2 ' where, 1^ is the second moment of left-view image and 

R is the second moment of the right-view image. 

Referring to FIG. 1 shows a typical stereoscopic video 
In practice, the various E() terms in the above equations imaging, processing and display system. A scene, 100, is 
can be replaced by the statistical moments of histograms. We 25 captured by a stereoscopic camera composed of individual 
use notation I^, 1^, 1^ and I^ to respectively denote the camera or lenses 101 and 102, where camera 101 generates 
first left moment, the first right moment, the second right left-view of the scene and camera 102 generates the right- 
moment and the joint moment between corresponding left- view of the scene. In a stereoscopic video system, the 
and right-images. With this substitution, the following equa- left-view is intended for the left-eye and the right-view is 
tions result: b^I^-al^, and 30 intended for right-eye of a human viewer. The left- and 

right-views at lines 103 and 104 are digitized, gain and offset 
iiLF-hJiR mismatch corrected and coded for efficient representation 

a ~ hR-Vw) 2 ' f° r transmission or storage and are then decoded for display. 

The operations of encoding, transmission/storage and 
35 decoding are represented by 105, at whose output, the 
One way to compute moments 1^, 1^, and I^ is via decoded left-view and the right-views are available on lines 
use of histograms, another way is to directly compute these n0 and m There are lwo ways of me output, 

values without reference to histograms. Since we use his- me first one> {imc multiplexed with active viewing LCD 
togram based approach in the next method, the necessary shuUer eyeglasses, and the second one, time simultaneous 
equations for histogram based approach are: 40 with passive polarizing eyeglasses. In display option 1, the 

left- and the right-views on lines 112 and 113 are time 
IlL - ^j x uP{xu)\ ^ multiplexed sequentially in 114 to form a sequence output at 

' 115 composed of left-view frame, right-view frame and so 

1 _ y f v (") on ' ^ e s *S na l on une is input to a stereo-ready monitor 
i» - 2j*«Pi**i). 45 or -r*v receiver 116 which uses a shutter synchronizer 117 to 

control timing of when the left and right views are presented 
i ZK = ^x\ip{x Rl y, ^ to human visual system by LCD shuttering glasses 118. In 

the display option 2, the left and right views are input to 
and projectors 122 and 123 over lines 120 and 121, respectively. 

50 Projectors 122 and 123 have polarizing filters 124 and 125 
hut -^xuxftipixuxfti), t,v) placed on lenses, these polarizers are either both linear or 

both circular and are matched to the passive eyeglasses 126, 
127 and 128. Thus, if linear polarizers are chosen, the filter 
where p(x,) represents the frequency of occurrence of event 124 on projector 122 is linear and its polarization direction 
x, here 1 takes values in 0 to 255 range and x, is same as J. 55 is orthogonal to that of filter 125 on projector 123 that is also 

linear polarized. Likewise, the left lens and the right lens of 
Method 3 passive eyeglasses 126, 127, 128 have linear polarizing 

This method is based on matching the geometrical prop- filters that are orthogonal, respectively. The image of scene 
erties of left- and right-view histograms and implicitly 100 is formed on screen 130 which has metallic content to 
minimizes error unlike the second method. Now to deter- 60 preserve the direction of polarization, 
mine gain, a,, and offset, b, by this method, we require that In FIG. 2, we show a generalized block diagram of a 
first and second moments of left-view and corrected right- compatible stereoscopic codec employing gain and offset 
view images to be equal. estimation and correction as a preprocessing operation. The 

Equaling the first moments, E(x jL )-aE(x /) )-b, from which left- view on line 103 is input to Left View Encoder, 201, via 
offset, b, can be rewritten as, b-ECxJ-aE^x^. Similar to 65 line 200. This Encoder outputs a coded bitstream on line 202 
method 2, the E( ) terms can be replaced by the appropriate and locally reconstructed frame on line 203. The right-view 
statistical moments, l 1L or 1^, resulting in b-I^-al^. is available on line 104 and is input on line 219 to gain 
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correction in Global Gain and Offset Correction Preproces- is now output by DC, 337 on line 338 and is utilized by 
sor (GOCP). 223, which outputs gain corrected right- view Decoder, 340, to reconstruct right-view frames which are 
frames on tine 214 which forms the input to a Right View output on line 341. The Decoder, 340 and DC, 337, together 
Encoder with Disparity Compensation, 215, which outputs are referred to as Motion Compensated and Disparity Corn- 
coded bitstream on line 216. Both the bitstreams on lines 5 pensated DCT Decoder, 342. As an example, MPEG-2 
202 and 216 are multiplexed by a System Multiplexer and Temporal scalability based, Temporal Enhancement 
packetized for transmission or storage on channel 225. At Encoder Decoder described in the above-referenced 
tie receiver, a System Demultiplexer, 216, unpacketizes and J Qm ?™\ Scalable Video Coding, article, can 

separates each of the bitstreams and offers on line 230, be **f d . of Since 15 c ? nsid - 

, £ . , , , _ . j > -pi * A ,„ erable nexibdity in the codec structure described, in ruture, 

left-view coded bitetream to Left view Decoder, 231 and 10 R an MPEG-4 coder optimized for coding of generic 

nght-view coded bitstream on line 235 to Right View be j d fof , eft . view and either an MpEG ^ 

Decoder with D^anty Compensation, 240. Both the Right ^ bascd cnaanccment codel or an 

View Encoder, 215, and the Right View Decoder, 240 use , lnr - . , ,. . , f ,. r _ 

°^ , . „. , . ... ' , MPEG-4 coder optimized for coding of noncompatible 

decoded left view signals, 203, used in Right View Encoder . • • «. j c j- c ■ u. • 

,.*.=,_ ' •. .• .• , stereoscopic view may be used for coding of right-view, 

and 238 used in Rich t View Decoder for disparity estimation is m/- >i l i c hli. ji.-T _ e 

" . % .. .. , ft , 111 FIG. 4 shows examples of well behaved histograms of 

and compensation. Earlier, we menUoned that GOCP, 223 ^ rj ht . vicw of stereoscopic vidco . In FIG. 4A 

corrects for gam and offset mismatch of nght-view with ^ ^ view Hjs ^ shows distinct charicteristic 

respect to left-view. It uses gain parameter, a and offset such as end ^ 4,,! ^ 403 or/and othcr charac . 

parameter, b, both on bne 222 for correction of m^match ^ ^ , ocations of ^ ^ ^ 

These parameters are confuted in Global Gam and Offset 20 maxim , 

Estimation Preprocessor (GOEP), 221 which takes as input, minimas • be * and ^ fac ^ ag charac(er . 

left-view frames on line 217 and nght-view frames on hne M m(s ^ ^ * fa In nG 4fi ^ ^ vjew 

22 ° i nd L ou ' pUU 0D *"» 2 , 22 ?? d ° &et f P arame,ers - . Histogram, 450, shows distinct characteristic poinU such as 

FIG. 3 shows an example block diagram of a stereoscopic e » ^ and 453 ^ ^ characteristic - mts 

yideo codec of this invention and is based on the basic two 25 ^ ^ of loba , and , oca] maxi 453 

layer codec structure explained in FIG. 2. The main differ- and 4M M befo maxi , ocaJ minimas 

ence wiu! respect to FIG. 2 is mat a specmc form 0 encoder abo b6 ^ ^ fee ^ ^ characteristic mts . 

and decoder is shown as an example for coding of left view From p , G 4A ^ FIQ 4B we caQ cQnclude ^ ^ are 

The encoder for coding left-view is a Motion Compensated ^ ^ m ri ^ t . view ^ ^ 

DCT Encoder, 301 and the co^ponding decoder is 30 corresponding distinct features of left-view his- 

Motion Compensated DCT Decoder, 331. To be more r & 

specific, an MPEG-1 or a nonviable MPEG-2 encoder/ a sim Med block di of one „ ample 

decoder could be used for coding of eft-view. The left-view f Gain ^ offse , Preprocessor (GOEP), 500. 

is input to encoder on hne 301 via hne 300. Tnis Encoder ^ k fa based m Mcthod j fof d estimatcs 

outpute a coded B.tetream on hne 302 and locally recon- 35 of dnan ^ offsetandthcG0 EP of 500 is a straightforward 

structed frame on line 303. The gam corrected nght-v.ew >s im ^ lementation of ^ obal m and o£feet mismatcn calcu . 

applied via line 314 to Motion and Disparity Compensated ] a tf on s 

DCT Encoder, 318, which outputs on line, 316, coded 

Bitstream representing the right-view. The encoder 318 xu-x^ 

consists of a modified Motion Compensated DCT Encoder 40 a = — - — and b = x u -ax R2 . 

315, Disparity Estimator (DE), 307, and a Disparity Com- RJ * 2 

pensator (DC), 312. The left-view frames become available 

on line 303 and are used by DE, 307, which also uses gain Assuming that location of two unique feature points x L1 and 
and offset corrected right-view frames via line 306. The x LZ , in left -view histogram that match corresponding feature 
resulting disparity estimates are sent over line 308 and then 45 points x R1 and Xj^ in right- view histogram have been 
via line 310 to DC, 312, which outputs a disparity compen- identified. The location of left view feature points are input 
sated signal on line 313. On a block by block basis, disparity on lines 501 and 502 whereas the location of right-view 
estimates, also called vectors are sent to encoder 315 and are feature points are input on lines 505 and 506. Differencer 
sent in the Bitstream output on line 316. Note that encoder 503 differences input on lines 501 and 502 and outputs the 
315 is a B-picture like encoder with 3 modes and can either 50 result on line 504 which forms one input to the divider 
choose disparity compensation, motion compensation or an element DIV, 509. Likewise, Differencer, 507, differences 
average of disparity and motion compensation on a block- input on lines 505 and 506 and outputs the result on line 508, 
by-block basis. The internal operations of Encoders, 301 and which forms the second input to divider DIV, 509, at whose 
315, and that of Decoders 331 and 340 are exactly identical output, is 511 is the gain parameter, a. The second feature 
to those described earlier in the above-referenced "Stereo- 55 points in left-view histograms is input via lines 513 to 
scopic Video Compression using Temporal Scalability" differencer 517, whereas, the second feature point in right- 
article and in above-indicated co-pending U.S. patent appli- view histogram is input via line 514 to multiplier device 
cation Ser. Nos. 08/452,463 and 08/452,464. The left and MULT, 515, where it is multiplied by gain, a, available on 
right bitstreams are multiplexed in Sys Mux, 324, sent over line 512 and the result output on line 516. Line 516 forms 
channel 325 and demultiplexed by Sys Demux 326 and 60 the second input to the differencer 517, at whose output 517 
offered to left- and right-view decoders. Line 330 carries the is the offset parameter, b. FIG. 6 shows a flowchart that can 
left-view coded bitstreams which are decoded by Motion be used to locate distinct feature points that match in the left- 
Compensated Decoder 331 and output on line 333. The and right-view histograms. Locate feature points may 
reconstructed lel'L-frames are employed by DC, 337, which require human interventions and may be difficult to auto- 
also uses decoded disparity vectors on line 336-. these 65 mate in some cases. In other cases, when well behaved 
vectors were obtained by modified Motion Compensated histograms exist like that in FIG. 4A an FIG. 4B, locating 
Decoder 340 via line 335. The disparity compensated signal feature points is not difficult. In any event, an iterative 



10/03/2001, EAST Version: 1.02.0008 



6,i: 

11 

procedure is probably the best one to try to locate feature 
points. The left-view is input via line 600 and the right-view 
input via line 601 to a processing unit, 602, that computes 
separate histograms of left and right-views. These histo- 
grams are analyzed in processing unit 604 to determine 
location of endpoint pairs in left-and right-histograms. In an 
analysis unit 606 it is determined if distinct endpoints have 
been found, and if so, the endpoints are saved, 630 and the 
processing concludes in 632. If analysis unit 606 determines 
that endpoint are not distinct, then a search for two peaks in 
processing unit 609 is carried out to find global peak 
(maxima) and local maxima in each of the two histograms 
computed in 602. Again, an analysis unit 611 determines if 
matching global and local maxima have been found in two 
histograms or not, and if they have been found, location of 
peaks are saved, 628, and the processing concludes in 632. 
Again, if required pair of matching maximas are not found, 
processing continues in unit 614 to determine a maxima and 
a minima in the two histograms. Next, analysis unit 619 
decides if the procedure of 614 has been a success or not, 
and if yes, location of peaks is saved, 626, and the process- 
ing stops, 632. If analyzing unit 616 determines that a 
matching peak and minima have not been found between the 
two histograms, a search for two matching minimas in the 
two histograms is carried in 619, followed by analysis in 
621, resulting in saving the location of pair of matching 
minimas, 624. Either way search is concluded in 632, and if 
no matching feature point pair has been found only then 
Method 2 or Method 3 (to be discussed) are tried. This 
procedure is separately followed for luminance and the two 
chrominance components. 

FIG. 7 shows Global Gain and Offset Correction Prepro- 
cessor (GOCP), 700. The right-view is input on line 701 and 
forms one input to multiplier MULT, 703, at whose other 
input 702 is the gain factor, a. The output of MULT, 703, on 
line 704 is input to adder 706, at whose other input, 705, is 
the offset, b. The resulting gain and offset corrected right- 
view is output on line 707. This procedure is separately 
followed for luminance and the two chrominance compo- 
nents. 

FIG. 8 shows a simplified block diagram of another 
example of Gain and Offset Estimation Preprocessor 
(GOEP), 800. This example is based on Method 2 for 
computing estimates of gain and offset and the GOEP 800 is 
a straightforward implementation of equations b-I^-al^, 
and 

a= Iir-Vir) 2 ' 

Left-view is input on line 801 and right-view on line 814, 
and are used by 805 to compute joint moment between left 
and right views, whose output is available on line 809 and 
forms one input to a differencer, 810. Simultaneously, left- 
and right- views are also used by 818 and 821 to respectively 
compute the first moments of left and right images. The 
output of 818 on line 819 and the output of 821 on line 822 
form two input to multiplier MULT, 823, at whose output, 
824, we have product of first moments of left- and right- 
views. This product forms the second input to differencer 
810, whose output, 811, forms the first input to divider DI V, 
812. Also, simultaneously, second moment of right- view .is 
calculated in 828 and output via line 831 and forms one 
input to differencer 840. Furthermore, simultaneously, in 
833 and 836 first moments of right-view are calculated and 
output on respective lines 834 and 837, which form the two 
inputs to MULT, 838, and outputs the product on line 839, 
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the output on line 839 forms the second input to the 
aforementioned differencer 840, which outputs the differ- 
ence on line 841. Incidentally, line 841 is the second input 
to the aforementioned divider DIV, 812 which finally out- 

5 puts gain parameter, a, on to line 845. The offset parameter, 
b is computed next and uses the gain parameter, a, just 
computed. The left-view and right-view first moments are 
again calculated (alternatively precomputed moments from 
previous calculations could be used) in 853 and 854. Output 

10 of 853 is 855 and feeds one input of the differencer, 860. The 
output of 854 is available on line 856 and is multiplied in 
858 with gain parameter, a, now available on line 846; the 
resulting output on line 859 forms the second input to 
differencer 860, which outputs offset parameter, b, on line 

15 861. 

FIGS. 9A, 9B, 9C and 9D show detailed block diagrams 
of computation of first moments, second moments and joint 
moments necessary in FIG. 8 (and later in FIG. 10). FIGS. 
9A, 9B, 9C and 9D are corresponding straightforward 
20 implementations of the statistical moment calculation equa- 
tions 

= 2*ziPC*u)l hit = £**iP(*jfi); hR = £*«P(*m); and 
25 „ 



In FIG. 9A, left-view is input on line 901 from which a 

30 histogram of levels is computed in device 903. A counter, 
905, which takes values from 0 to 255, outputs a level, 
one-at-a-time on line 906 which is used to index the histo- 
gram in 903. The output 904 of the histogram computing 
device 903 is the relative frequency of occurrence of the 

35 level on line 906. Quite simultaneously, the level on line 906 
is also applied on line 907 to a lookup table, 908, which 
outputs on line 916, a value same as input value on line 907. 
Both lines 904 and 916 are inputs to a multiply accumulator 
MULT ACC, 917 which repeats the loop for all values of 

40 counter before outputting the result on line 918. 

Next the operation of FIG. 9B is now described in which 
right-view is input on line 921 from which a histogram of 
levels is computed in 923. A counter, 925, which takes 
values from 0 to 255, outputs a level, one-at-a-time , on line 

45 926 which is used to index the histogram in device 923. The 
output on line 924 is the relative frequency of occurrence of 
the level on line 926. Quite simultaneously, the level on line 
926 is also applied on line 927 to a lookup table, 928, which 
in our case outputs on line 936, a value same as input value 

50 on line 927. Both lines 924 and 936 are inputs to a multiply 
accumulator MULT ACC, 937 which repeats the loop for all 
values of counter before outputting the result on line 938. 

Now, the operation of FIG. 9C is described in which the 
right-view is input on line 941 from which a histogram of 

55 levels is computed in device 943. A counter, 945, which 
takes in values from 0 to 255, outputs a level, one-at-a-time 
on line 946. Quite simultaneously, the level on line 946 is 
also applied on line 947 to a look-up table, 948, which in our 
case outputs on line 949, a value same as input value on line 

60 947. Line 949 is applied to both inputs 950 and 951 of a 
multiplier MULT, 952, whose output on line 956, as well as 
the value on line 944 is applied to a multiplier accumulator 
MULT ACC, 957 which repeats the loop for aD values of 
counter before outputting the result on liuc 958. 

65 Next, the operation of FIG. 9 D is now described in which 
the left- view is input on line 961 and the right- view is input 
on line 962 from which a combined histogram of levels is 
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computed in 963. A counter, 965, which takes in values from 
0 to 255, outputs a level, one-at-a-time on line 966. Quite 
simultaneously, the level on line 966 is also applied on line 
967 to a look-up table, 968, which in our case outputs on 
lines 969 and 971, a value same as input value on line 967. 
Line 969 is applied as one input and line 971 as another input 
to multiplier MULT, 972, whose output on line 976, as well 
as the value on line 964 is applied to a multiplier accumu- 
lator MULT ACC, 977 which repeats the loop for all values 
of counter before outputting the result on line 978. 

FIG. 10 shows a simplified block diagram of yet another 
example of Gain and Offset Estimation Preprocessor 
(GOEP), 1000. This example is based on Method 3 for 
computing estimates of gain and offset. GOEP of 1000 is a 
straightforward implementation of gain and offset equations 



V/ 2 «-</i«) 2 

and b=I iL -aI LR , respectively. There is considerable similar- 
ity in general operation of the Gain and Offset Estimation 
Preprocessor illustrated in FIG. 10 and that illustrated in 
FIG. 8. Left- view and right -views are input on lines 1001 
and 1014. Line 1001 inputs left-view to second moment 
computer 1005 via line 1002, whose output is available on 
line 1009 and forms one input to a differencer 1110. Left- 
view and right -view are also input to respective first 
moments computers, 1018 and 1021, whose corresponding 
outputs, 1019 and 1021 form inputs to multiplier MULT, 
1023. This multiplier outputs product on line 1024, which 
forms the other input to the aforementioned differencer, 
1010, the difference is output on line 1011 and forms one 
input to divider DIV, 1012. Simultaneously, right-view is 
also applied to a second moment computer, 1028, and to two 
first moment computers, 1033 and 1036. The output of 1028 
is available on line 1031 and forms one input to differencer 
1040. The output of 1033 and 1036 on respective lines 1034 
and 1037 is input to a multiplier 1038, whose output on line 
1039 forms the second input to aforementioned differencer 
1040, whose output on line 1041 forms the second input to 
the aforementioned divider DIV, 1012. The output of divider 
1012 on line 1013 enters SQRT function in 1042. The output 
of SQRT is gain parameter, a, available on line 1045. 
Further, left and right-views are also input to first moment 
computers, 1053 and 1054. The output of 1053 on line 1055 
forms one input to differencer 1060. The output of 1054 on 
line 1056 is multiplied in MULT, 1058 with gain parameter, 
a, available on line 1046. The output of MULT on line 1059 
forms the second input to the aforementioned differencer, 
1060, at whose output on line 1061 is the offset parameter, 
b. 

FIG. 11 A shows block diagram of stereoscopic codec 
structure that estimates gain and offset parameters and 
performs mismatch correction after coding at the decoder/ 
display processor prior to display. Left-view is input on line 
103 to Left View Encoder, 1101, which outputs coded 
left-view Bitstream on line 1122 and decoded left- view 
frames on line 1103. The right- view is input on line 104 to 
Right View Encoder with Disparity Compensation, 1115, at 
whose output 1116 is the coded right-view Bitstream. The 
two bitstreams are multiplexed in Sys Mux, 1124 and sent 
over channel, 1125 to Sys Demux, 1126, which demulti- 
plexes and outputs individual bitstreams of left- and right- 
view on lines 1130 and 1135. Lines 1130 and 1135 respec- 
tively feed Left View Decoder, 1131 and Right View 
Decoder, 1140. The output of 1131 is decoded left- view 
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frames and are input via line 1138 to Right View Decoder, 
1140. The output of 1131 via line 1143 and the output of 
1140 via fine 1142 are applied to Global Gain and Offset 
Estimation Postprocessor, 1144, which outputs gain and 

5 offset values on line 1144 to Global Gain and Offset Post- 
processor 1147, at the output of which, 1148, is the gain and 
offset corrected decoded right-view. The operation of 1147 
is identical to the operation of GO CP, 223 and the operation 
of 1144 is exactly identical to operation of GOEP, 221. Any 

10 of the gain and offset correction methods described in 500, 
800 and 1000 can be used in 1144. Likewise, gain and offset 
correction described with respect to the GOCP 700 of FIG. 
7, can be used for Global Gain and Offset Postprocessor 
1147. 

15 FIG. 11B shows block diagram of stereoscopic codec 
structure that estimates gain and offset parameters at the 
preprocessor prior to coding but performs mismatch correc- 
tion at the decoder/display processor prior to display. Left- 
view is input on line 103 to Left View Encoder, 1153, which 

20 outputs coded left-view bitstream on line 1152 and decoded 
left-view frames on line 1153. The right-view on line 104 is 
input via line 1170 to Global Gain and Ofiset Estimation 
Postprocessor, 1171, at the other input to which is left-view 
signal applied via line 1167. The output of 1171 is gain and 

25 offset parameters on line 1172 and are encoded by Right 
View Encoder, 1115. Also, the right- view on line 1155 is 
input to 1165, at the output of which, 1166, is the coded 
right-view Bitstream. The two bitstreams are multiplexed in 
Sys Mux, 1174 and sent over channel, 1175 to Sys Demux, 

3 q 1176, which demultiplexes and outputs individual bitstreams 
of left- and right -view on lines 1180 and 1185. Lines 1180 
and 1185 respectively feed Left View Decoder, 1181 and 
Right View Decoder, 1190. The output of 1181 is decoded 
left-view frames and are input via line 1188 to Right View 

35 Decoder, 1190. The output of 1190 via line 1191 is applied 
to Global Gain and Offset Correction Postprocessor, 1193, 
which also needs global gain and offset parameters on line 

1192 decoded by 1190 from the Bitstream. The output of 

1193 is the gain corrected decoded right-view. The operation 
w of 1193 is identical to the operation of GOCP, 223 and the 

operation of 1171 is exactly identical to operation of GOEP, 
221. Any of the gain and offset correction methods described 
in 500, 800 and 1000 can be used in 1171. Likewise, gain 
and offset correction described with respect to the GOCP 

^ 5 700 can be used for 1193. 

It should be understood that the preferred embodiments 
and examples described are for illustrative purposes only 
and are not to be construed as limiting the scope of the 
present invention which is properly delineated only in the 

50 appended claims. 

What is claimed is: 

1. An apparatus for correcting global mismatch in gain 
between first and second views forming a stereoscopic 
image comprising: 
55 a) means for generating a first histogram of said first view 
and a second histogram of said second view, each said 
first and second histogram representing a frequency of 
occurrence of a characteristic component of said first 
and second views, respectively; 
60 b) means for comparing said first histogram of said first 
view with said second histogram of said second view 
and estimating mismatch in gain, a, between said first 
and second views; and, 
c) means for correcting the first and second views of said 
65 stereoscopic image by multiplying said mismatch in 
gain a and a second signal representing the second view 
and generating a gain corrected second signal of said 
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stereoscopic image such that said gain corrected second 
signal and a first signal representing the first view form 
a stereoscopic image of improved quality. 

2. The apparatus according to claim 1, wherein said first 
view and said second view are digitized representations of 5 
left and right views of the stereoscopic image, respectively, 
said characteristic component of said first and second views 
being luminance amplitude levels of said left and right 
views. 

3. The apparatus according to claim 1, wherein said first JQ 
view and said second view are digitized representations of 
left and right views of tee stereoscopic image, respectively, 
said characteristic component of said first and second views 
being color component levels of said left and right views. 

4. The apparatus according to claim 1, wherein said means 
for comparing said first histogram of said first view with said 15 
second histogram of said second view includes means for 
obtaining a first set of characteristic points, x xi , x i2 , from 
said first histogram, and obtaining a second set of charac- 
teristic points, x R19 from said second view, said second 
set of points substantially corresponding to said first set of 20 
points, further including means for calculating the mismatch 

in gain, a, according to 

a = . 

Xr/ - x R2 25 

5. The apparatus according to claim 4, wherein said first 
set of characteristic points, x L1 , x L29 of said first histogram 
is a pair selected from the group consisting of: distinct 
endpoints of said first histogram, first and second maxima 30 
extrema points of said first histogram, a maxima extrema 
point and a local minima point of said first histogram, and 
first and second local minima points of said first histogram. 

6. The apparatus according to claim 5, further including 
means for estimating and correcting global mismatch in 35 
oflset, b, between said first and second views including 
means for calculating b=x L2 -ax KZ , and, means for adding 
said oflset b to said gain corrected second signal and 
generating a gain and offset corrected second view of said 
stereoscopic image. 40 

7. The apparatus according to claim 1, wherein said means 
for comparing said first histogram of said first view with said 
second histogram of said second view includes means for 
calculating a statistical first left moment, 1^, of said first 
histogram, a statistical first right moment, 1^, of said second 45 
histogram, a statistical second right moment, I^, of said 
second histogram, and a statistical joint moment, 
between said first and second histograms; said statistical first 
left moment, 1^, being calculated as 

50 



said statistical first right moment, 1^, being calculated as 



said statistical second right moment, 1^, being calculated as 60 
h* - 

65 

and, said statistical joint moment, between said first and 
second histograms being calculated as 



*2L* = / ^UXRiPjXLiXfti) 



where p(x^ represents the frequency of occurrence of the 
characteristic component at x,. 

8. The apparatus according to claim 7, wherein said means 
for estimating mismatch in gain, a, includes means for 
calculating 

_ hm - hUiR 

9. The apparatus according to claim 8, further including 
means for estimating and correcting global mismatch in 
oflset, b, between said first and second views including 
means for calculating b^I^-al^ and, means for adding said 
mismatch in oflset b to said gain corrected second view and 
generating a gain and oflset corrected second view of said 
stereoscopic image. 

10. The apparatus according to claim 7, wherein said 
means for comparing said first histogram of said first view 
with said second histogram of said second view includes 
means for computing a statistical second left moment, I^, of 
said second histogram by calculating 



wherein said means for estimating mismatch in gain, a, 
includes means for calculating 

11. The apparatus according to claim 10, further including 
means for estimating and correcting global mismatch in 
oflset, b, between said first and second views including 
means for calculating b-I^-al^ and, means for adding said 
mismatch in oflset b to said gain corrected second view and 
generating a gain and oflset corrected second view of said 
stereoscopic image. 

12. An apparatus for increasing compression efficiency of 
a stereoscopic image encoder employed for compressively 
encoding first and second signals representing respective 
first and second views of a stereoscopic image, said appa- 
ratus comprising: 

a) means for generating a first histogram of said first view 
and a second histogram of said second view, said first 
and second histograms representing a frequency of 
occurrence of a characteristic component of said first 
and second views, respectively; 

b) means for comparing said first histogram of said first 
view with said second histogram of said second view 
and for estimating mismatch in gain, a, and mismatch 
in offset, b, between said first and second views; and, 

c) means for multiplying said mismatch in gain a to said 
second signal to obtain a gain corrected second signal 
for said stereoscopic image and means for adding said 
mismatch in oflset b to said gain corrected second 
signal to obtain a gain and offset corrected second 
signal, 

whereby compression efficiency is increased when said 
gain and offset corrected second signal and said first 
signal are input to the stereoscopic image encoder. 
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13. The apparatus according to claim 12, wherein said first 

view and said second view are digitized representations of iuR - ^x u x Ri p{x u x Ri ) 

left and right views of the stereoscopic image, respectively, i 
said characteristic component of said first and second views 

being luminance amplitude levels of said left and right 5 wnere p ( x .) represents the frequency of occurrence of the 

views. characteristic component at x f . 

14. The apparatus according to claim 12, wherein said first 19. The apparatus according to claim 18, wherein said 
view and said second view are digitized representations of means for estimating the mismatch in gain, a, includes 
left and right views of the stereoscopic image, respectively, 10 means for calculating 

said characteristic component of said first and second views 
being color component levels of said left and right views. 

15. The apparatus according to claim 12, wherein said 
means for comparing said first histogram of said first view 

with said second histogram of said second view includes 15 20. The apparatus according to claim 19, wherein said 

means for obtaining a first set of characteristic points, x^, means ^ x estimating the mismatch in offset b between said 

f . 1 , . • • , e first and second views odes means for calculating b=I lf - 

x R1 , from said first histogram, and obtaining a second set or a j 

characteristic points, x^, x B2 , from said second histogram, 21 a p paratus according to claim 18, wherein said 

said second set of points substantially corresponding to said 20 means for comparing said first histogram of said first view 

first set of points, further including means for calculating the with said second histogram of said second view includes 

mismatch in gain, a, according to means for computing a statistical second left moment, L^, of 

said second histogram by calculating 

16. The apparatus according to claim 15, wherein said first 

set of characteristic points, x^ J7 x i2 , of said first histogram is sa * d means for estimating mismatch in gain, a, includes 

a pair selected from the group consisting of: distinct end- 30 meaDS for calculatin S 
points of said first histogram, first and second maxima 

extrema points of said first histogram, a maxima extrema a _ j !2L ~ ^il) 2 

point and a local minima point of said first histogram, and \'m - Vw) 2 
first and second local minima points of said first histogram. 

17. The apparatus according to claim 15, wherein said 35 22. The apparatus according to claim 21, wherein said 
means for estimating said mismatch in offset, b, between means for estimating mismatch in offset, b, between said 
said first and second views includes means for calculating first arid second views includes means for calculating 
b=x =ax b^^-al^. 

it 2 ™.* 2 4. _j* * 1 ■ 11 u • a 23. The apparatus according to claim 12, wherein the 

18. -Die apparatus according to claim 12, wherem sa.d 4Q an MpECM for enc ^ ding said firet 

means for comparing said first histogram of said first view sjgnal and afl MPEG _ 2 Temporal scalability encoder for 

with said second histogram of said second view includes encoding said gain and offset corrected second signal, 

means for calculating a statistical first left moment, 1^, of 24. The apparatus according to claim 12, wherein the 

said first histogram, a statistical first right moment, 1^, of encoder includes a non-scalable MPEG-2 encoder for 

said second histogram, a statistical second right moment, 45 encoding said first signal and an MPEG-2 Temporal scal- 

L^, of said second histogram, and a statistical joint moment, ability encoder for encoding said gain and offset corrected 

between said first and second histograms; said statistical second signal. 

first left moment, 1^, being calculated as 25 ' ™ e apparatus according to claim 12 wherein the 

encoder includes an H.263 encoder for encoding said first 

50 signal and an MPEG-4 utilizing disparity/motion compen- 

IlL = 2^ XuP ^ Xu ^ sated encoder for encoding said gain and offset corrected 

second signal. 

26. A method for estimating and correcting global mis- 
said statistical first right moment, 1^, being calculated as match in ^ md offset between first and second signals 

55 representing first and second views of a stereoscopic image 

^ comprising the steps of: 

hR = 2y*fiPta;); a ^ providing a first histogram of said first view and a 

second histogram of said second view, said first and 
second histograms representing a frequency of occur- 

said statistical second right moment, L^, being calculated as 60 ™acc of a characteristic component of said first and 

second views, respectively; 

_ y ; b) comparing said first histogram of said first view with 
2 * ~ £f* RiF XR ' ' said second histogram of said second view and esti- 
mating mismatch in gain, a, and mismatch in offset, b, 
65 between said first and second views; 
and, said statistical joint moment, Ijij? between said first and c) correcting the first and second views of said stereo- 
second histograms being calculated as scopic image by multiplying said mismatch in gain a 
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and said second signal to obtain a gain corrected second said statistical second right moment, I^, being calculated as 
signal for said stereoscopic image and adding said 

mismatch in offset b to said gain corrected second j 2R = £x* Rl p{x K )\ 

signal to obtain a gain and offset corrected second / 

signal such that when said gain and offset corrected 5 

second signal and said first signal arc combined, a alK , >sl i d8tati ^j ointmom e n ^i^betwe«is^rtataiiil 

stereoscopic image of unproved quality is formed. ^ loffam& being calculated „ 

27. The method according to claim 26, wherein said first 
view and said second view are digitized representations of 
left and right views of the stereoscopic image, respectively, 
said characteristic component of said first and second views 
being luminance amplitude levels of said left and right 

views, where p(x i ) represents the frequency of occurrence of the 

28. The method according to claim 26, wherein said first characteristic component at x ( . 

view and said second view are digitized representations of 33 - ^ method according to claim 32, wherein said 

left and right views of the stereoscopic image, respectively, mismatch in gain, a, is estimated by calculating 
said characteristic component of said first and second views 

being color component levels of said left and right views. a = Ilut ~ . 

29. The method according to claim 26, wherein said step 20 ' 2 * ~ ^ llR ^ 
(b) of comparing said first histogram of said first view with 

said second histogram of said second view includes finding 34. The method according to claim 33, wherein said step 

a first set of identifiable characteristic points, x Llt x i2 , from of estimating mismatch in offset, b, includes calculating 

said first histogram, and finding a second set of characteristic b-I^-al^. 

points, x Rl , x^, from said second histogram, said second set 25 35. The method according to claim 32, wherein said step 

of points substantially corresponding to said first set of (b) of comparing said first histogram of said first view with 

points, and wherein estimating mismatch in gain, a, includes said second histogram of said second view includes the step 

calculating comparing geometric shapes of said first and second 

histograms by further computing a statistical second left 

Xu - Xl2 30 moment, I^, of said second histogram by calculating 



30. The method according to claim 29, wherein said first 
set x Ll , x L2 , of characteristic identifiable points from said 35 

first histogram a pair selected from the group consisting of: and estimating said mismatch in gain, a, by calculating 
distinct endpoints of said first histogram, first and second 

maxima extrema points of said first histogram, a maxima / fa -(hi? 

extrema point and a local minima point of said first a = \ SjR _ y lR f * 

histogram, and first and second local minima points of said 40 

first histogram. ^ ^ tJ , . „^ , 

31. The method according to claim 29, wherein said step J** method according to claim 35, wherein said step 
of estimating mismatch in offset, b, between said first and ^f^*** m * m * ich m offset ' b ' includes calculatin g 

second includes calculating b — x ax at' 

„„ ^ , . . z ? • ^ , . . A 45 37. The method according to claim 26, further including 

32 The method according to claim 26, wherein said step ^ of essivel * ncodi ^ first and m ai J 

(b) of comparing said first histogram of said first view with qS&c{ si ^ for transrmssion or stocage ^ 

said second histogram of said second view includes the step a compressed state> and deco ding said compressed fist and 

of: applying a technique of linear minimization of mean comp ressed gain and offset corrected second signals to form 

square error estimates between first and second views by 5Q me stereoscopic image, wherein said method steps a), b) and 

computing a statistical first left moment, 1^, of said first c ) are performed prior to encoding to improve coding 

histogram, a statistical first right moment, 1^, of said second efficiency. 

histogram, a statistical second right moment, L^, of said 38. The method according to claim 26, further including 
second histogram, and a statistical joint moment, I21A the steps of compressively encoding a first and second 
between said first and second histograms; said statistical first 55 original signals for transmission or storage in a compressed 
left moment, \ lL , being calculated as state, and decoding said compressed first and second origi- 

nal signals to form said first and second signals, wherein said 
hi. ~ Y *up{xu) method steps a), b) and c) are performed after decoding of 

V 1 said first and second original signals to provide a stereo- 

60 scopic image of improved quality. 

39. A method for increasing compression efficiency of a 
said statistical first right moment, 1^, being calculated as stereoscopic image encoder employed for compressively 

encoding first and second signals representing respective 
fiR _ y x Ri p{x Ri y, ^ irst anc * secor id views of a stereoscopic image comprising 

i ' " 65 the steps of: 

a) providing a first histogram of said first view and a 
second histogram of said second view, said first and 
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second histograms representing a frequency of occur- 
rence of a characteristic component of said first and ijt _ y Xu p^e u ) t 
second views, respectively; / 

b) comparing said first histogram of said first view with 

said second histogram of said second view and esti- s ^ statistical first right moment, 1^, being calculated as 
mating mismatch in gain, a, and mismatch in offset, b, 

between said first and second views; , v , * 

c) correcting said stereoscopic image by multiplying said <■ 
gain a to said second signal to obtain a gain corrected jq 

second signal for said stereoscopic image and adding ^ statistical second ri ht momeQt) ^ ^ calculate d as 

said offset b to said gain corrected second signal to 

obtain a gain and offset corrected second signal, « 

wherein said gain and offset corrected second signal is 2R y 1 *' K * 

obtained prior to encoding to increase encoding efE- 15 

ciency. 

Ai\ m. *u j j ■ , , • 1ft . . j and. said statistical joint moment, U, R between said first and 

40. The method according to claim 39. wherein said first > , . . J , . t i ' j 

.. ...... A . * second histograms being calculated as 

view and said second view are digitized representations of 

left and right views of the stereoscopic image, respectively, ^ 

said characteristic component of said first and second views ^ ~ 2a x uxiapVcux K ) 

being luminance amplitude levels of said left and right ' 

views. 

41. The method according to claim 39, wherein said first where p(x-) represents the frequency of occurrence of the 
view and said second view are digitized representations of characteristic component at x ; . 

left and right views of the stereoscopic image, respectively, 46. The method according to claim 45, wherein said 
said characteristic component of said first and second views mismatch in gain, a, is estimated by calculating 
being color component levels of said left and right views. 

42. The method according to claim 39, wherein said step imt- Wir 
(b) of comparing said first histogram of said first view with ^ a ~ i 2R -{i iR ? ' 
said second histogram of said second view includes finding 

a first set of characteristic points, x L1 , x,i, from said first __ , ^ . -j . 

histogram, and finding a second set of characteristic points, t 47 ' .The method according to claim 46, wherein said step 
x* 2 , from said second history said second set of points of m ^ tcb ™ ° ffset > b r elween said first and 

substantially corresponding to said first set of points, and s^d views includes calculating b-I^-aW 

wherein estimating the mismatch in gain, a, includes calcu- 35 «■ ™ e me,hod a .~° rdm S to claml f w ^ rem ^ 



lating 



(b) of comparing said first histogram of said first view with 
said second histogram of said second view includes the step 
of: comparing geometric shapes of said first and second 
histograms by further computing a statistical second left 
40 moment, I^, of said second histogram by calculating 

43. The method according to claim 42, wherein said first ^ _ y ^ ^ ^ 
set x L1 , x L2 , of identifiable points in said first histogram is a *f uP * u ' 
pair selected from the group consisting of: distinct endpoints 

of said first histogram, first and second maxima extrema 45 

points of said first histogram, a maxima extrema point and ™& estimating said mismatch in gain, a, by calculating a 

a local minima point of said first histogram, and first and 

second local minima points of said first histogram. / 

44. The method according to claim 42, further including a ~ yj t _ (/tft j 2 * 
the step of determining global mismatch in offset, b, between 50 

said first and second view includes calculating b=x i2 -ax J?2 

and, adding said offset b to said gain corrected second signal 49. The method according to claim 48, wherein said step 
to obtain a gain and offset corrected second signal such that of estimating mismatch in offset, b, between said first and 
when said gain and offset corrected second signal and said second views includes calculating b^j^-al^. 

first signal are combined, a stereoscopic image of improved 55 An apparatus for correcting mismatch in gain and 

quality is formed. offset between multiple views of a multi-view and a refer- 

45. The method according to claim 39, wherein said step ence view of the multi-view, the multiple views represented 
(b) of comparing said first histogram of said first view with by multiple signals, one view per signal, the reference view 
said second histogram of said second view includes the step represented by a reference signal, the apparatus comprising: 
of: applying a technique of linear minimization of mean 60 a) means for generating a reference histogram of the 
square error estimates between said first and second views reference signal and for generating multiple histograms 
by computing a statistical first left moment, \ 1L , of snid first from the multiple signals, ench histogram representing 
histogram, a statistical first right moment, 1^, of said second a characteristic component of corresponding signal; 
histogram, a statistical second right moment, 1^, of said b) means for comparing each of said multiple histograms 
second histogram, and a statistical joint moment, 65 with the reference histogram and for estimating mis- 
between said first and second histograms; said statistical first matches in gain and offset between each of said mul- 
left moment, \ lL , being calculated as tiple histograms and the reference histogram; 
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c) means for gain correcting each of said multiple signals 
by multiplying each of said multiple signals and cor- 
responding estimated mismatch in gain to obtain mul- 
tiple gain corrected signals; 

d) means for offset correcting each of said multiple gain 5 
corrected signals by adding to each of said multiple 



24 

gain corrected signals corresponding estimated mis- 
match in offset, thereby obtaining multiple offset and 
gain corrected signals that can be combined with the 
reference signal to produce a multi-view video of 
improved quality. 

* * * * * 
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